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Abstract. This paper presents a novel technique for state space reduc- 
tion of probabilistic specifications, based on a newly developed notion of 
confluence for probabilistic automata. We prove that this reduction pre- 
serves branching probabilistic bisimulation and can be applied on-the-fiy. 
To support the technique, we introduce a method for detecting confluent 
transitions in the context of a probabilistic process algebra with data, 
facilitated by an earlier defined linear format. A case study demonstrates 
that significant reductions can be obtained. 

1 Introduction 

Model checking of probabilistic systems is getting more and more attention, but 
there still is a large gap between the number of techniques supporting tradi- 
tional model checking and those supporting probabilistic model checking. Espe- 
cially methods aimed at reducing state spaces are greatly needed to battle the 
omnipresent state space explosion. 

In this paper, we generalise the notion of confluence [10] from labelled tran- 
sition systems (LTSs) to probabilistic automata (PAs) [14]. Basically, we define 
under which conditions unobservable transitions (often called r-transitions) do 
not influence a PA's behaviour (i.e., they commute with all other transitions). 
Using this new notion of probabilistic confluence, we introduce a symbolic tech- 
nique that reduces PAs while preserving branching probabilistic bisimulation. 

The non-probabilistic case. Our methodology follows the approach for LTSs 
from [4]. It consists of the following steps: (i) a system is specified as the parallel 
composition of several processes with data; (ii) the specification is linearised to 
a canonical form that facilitates symbolic manipulations; (iii) first-order logic 
formulas arc generated to check symbolically which T-transitions are confluent; 
(iv) an LTS is generated in such a way that confluent r-transitions are given 
priority, leading to an on-the-fly (potentially exponential) state space reduc- 
tion. Refinements by [12] make it even possible to perform confluence detection 
on-the-fly by means of boolean equation systems. 

The probabilistic case. After recalling some basic concepts from probability the- 
ory and probabilistic automata, we introduce three novel notions of probabilistic 
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confluence. Inspired by [3], these are weak probabilistic confluence, probabilistic 
confluence and strong probabilistic confluence (in decreasing order of reduction 
power, but in increasing order of detection efficiency). 

We prove that the stronger notions imply the weaker ones, and that r-transi- 
tions that are confluent according to any of these notions always connect branch- 
ing probabilistically bisimilar states. Basically, this means that they can be given 
priority without losing any behaviour. Based on this idea, we propose a reduc- 
tion technique using weak probabilistic confluence, which merges all states that 
can reach each other by traversing only confluent transitions. Additionally, we 
propose a reduction technique that can be applied using the two stronger notions 
of confluence. As opposed to the first technique it does not need to merge states; 
rather, it chooses a representative state that has all relevant behaviour. We prove 
that both reduction techniques yield a branching probabilistically bisimilar PA. 
Therefore, they preserve virtually all interesting temporal properties. 

As we want to analyse systems that would normally be too large, we need to 
detect confluence symbolically and use it to reduce on-the-fly during state space 
generation. That way, the unreduced PA never needs to be generated. Since we 
have not found an efficient method for detecting (weak) probabilistic confluence, 
we only provide a detection method for strong probabilistic confluence. Here, we 
exploit a previously defined probabilistic process-algebraic linear format, which is 
capable of modelling any system consisting of parallel components with data [9] . 
In this paper, we show how symbolic r-transitions can be proven confluent by 
solving formulas in first-order logic over this format. As a result, confluence can 
be detected symbolically, and the reduced PA can be generated on-thc-fly. We 
present a case study of leader election protocols, showing significant reductions. 

Related work. As mentioned before, we basically generalise the techniques pre- 
sented in [4] to probabilistic automata. 

In the probabilistic setting, several reduction techniques similar to ours exist. 
Most of these are generalisations of the well-known concept of partial-order re- 
duction (POR) [13]. In [2] and [5], the concept of POR was lifted to Markov 
decision processes, providing reductions that preserve quantitative LTL\X. This 
was refined in [1] to probabilistic CTL, a branching logic. Recently, a revision of 
POR for distributed schedulers was introduced and implemented in PRISM [7]. 

Our confluence reduction differs from these techniques on several accounts. 
First, POR is applicable on state-based systems, whereas our confluence reduc- 
tion is the first technique that can be used for action-based systems. As the 
transformation between action- and state-based blows up the state space [11], 
having confluence reduction really provides new possibilities. Second, the defini- 
tion of confluence is quite elegant, and (strong) confluence seems to be of a more 
local nature (which makes the correctness proofs easier). Third, the detection 
of POR requires language-specific heuristics, whereas confluence reduction acts 
at a more semantic level and can be implemented by a generic theorem prover. 
(Alternatively, decision procedures for a fixed set of data types could be devised.) 

Our case study shows that the reductions obtained using probabilistic con- 
fluence are comparable to the reductions obtained by POR [8]. 
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2 Preliminaries 



Given a set S, an element s G S and an equivalence relation R C S x 5, we 
write [s]_r for the equivalence class of s under _R, i.e., [s]r = {s' G S \ (s, s') G i?}. 
We write iS/i? = {[s]r \ s G 5} for the set of all equivalence classes in S. 

2.1 Probability theory and probabilistic automata 

Definition 1 (Probability distributions). A probability distribution over a 
countable set S is a function /i: S — > [0,1] such that X^sesM 5 ) = 1- Given 
S' C S, we write fi(S') to denote Xs'eS' m( s ')- ^ e Mse Distr(S) to denote the 
set of all probability distributions over S, and Distr*(S') for the set of all sub- 
stochastic probability distributions over S, i.e., where < X^esM 5 ) ^ 1- 

Given a probability distribution \i with /x(si) = Pi, ^(s 2 ) = P2, ■ ■ ■ (Pi 7^ 0), 
we write /j, = {s\ i-> pi, s 2 M> p 2 , . . . } and /ei spt(/x) = {s\, s 2 , • • • } denote its 
support. For the deterministic distribution fj, determined by fi(t) = 1 we write It. 

Given an equivalence relation R over S and two probability distributions /1, // 
over S, we say that \x =r \j! if and only if fi(C) = //(C) for all C G S/R. 

Probabilistic automata (PAs) are similar to labelled transition systems, ex- 
cept that transitions do not have a fixed successor state anymore. Instead, the 
state reached after taking a certain transition is determined by a probability 
distribution [14]. The transitions themselves can be chosen nondctcrministically. 

Definition 2 (Probabilistic automata). A probabilistic automaton (PA) is 
a tuple A — (S, s°, L, A), where S is a countable set of states of which s° G S is 
initial, L is a countable set of actions, and A C S x L x Distr(S) is a countable 
transition relation. We assume that every PA contains an unobservable action 
t G L. If (s,a, /i) G A, we write s ^ n, meaning that state s enables action a, 
after which the probability to go to s' G S is fi(s'). If \i = l t , we write s t. 

Definition 3 (Paths and traces). Given a PA A = (S,s°,L,A), we define a 
path of A to be either a finite sequence n — s ai ~^ 1 Si ° 2 -^> 2 s 2 a3 -^ 3 . . . a "w" s n , 

or an infinite sequence n = s ~3 Si ~S s 2 ~S 

For finite paths we require Sj G S for all < i < n, and Sj " i+1 > as well 

as /ij+i(sj+i) > for allO < i < n. For infinite paths these properties should hold 
for all i > 0. A fragment s a -& s' denotes that the transition s fi was chosen 
from state s, after which the successor s' was selected by chance (so n(s') > 0). 

rr a,l a , o,l S2 a,l s _ . . ,, . , , ., a 

— If 7T = s ~» si ~» ... s n is a path of A (n > 0), we write s — » s n . 
If each transition is also allowed to be faced backwards, we write Sq «-^» s n . 
If there exists a state t such that s ^» t and s' t, we write s -%<A s' . 

— We use prefix(ir,i) to denote so" 1 ^ 1 . . . a, -& % s i: and step(ir,i) to denote the 
transition (sj_i, dj, fii). When 7r is finite we define \n\ = n and last(jr) — s n . 

— We use finpaths A to denote the set of all finite paths of A, and finpaths^(s) 
for all finite paths where s = s. 

— A path's trace is the sequence of actions obtained by omitting all its states, 
distributions and T-steps; given tt = sq ^ si s 2 ... s n , we de- 
note the sequence 0,10,3 . . . a n by trace(n). 
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2.2 Schedulers 



To resolve the nondeterminism in probabilistic automata schedulers are used [16]. 
Basically, a scheduler is just a function defining for each finite path which transi- 
tion to take next. The decisions of schedulers are allowed to be randomised, i.e., 
instead of choosing a single transition a scheduler might resolve a nondetermin- 
istic choice by a probabilistic choice. Schedulers can be partial, i.e., they might 
assign some probability to the decision of not choosing any next transition. 

Definition 4 (Schedulers). A scheduler for a PA A — (S, s°, L, A) is a func- 
tion 

S: finpaths A -> Distr({_L} U A), 

such that for every 7r 6 finpaths A the transitions (s, a, /i) that are scheduled by S 
after ir (i.e., S(ir)(s, a, ji) > 0) are indeed possible after ir , i.e., s — last(ir). The 
decision of not choosing any transition is represented by _L. 

We now define the notions of finite and maximal paths of a PA given a 
scheduler. 

Definition 5 (Paths and maximal paths). Let A be a PA and S a scheduler 
for A. Then, the set of finite paths of A under S is given by 

finpaths A = {ir G finpaths A VO < i < \ir\ . S{prefix{'K, i))(step(n, i + 1)) > 0}. 

We define finpaths A (s) C finpaths^ as the set of all such paths starting in s. 
The set of maximal paths of A under S is given by 

maxpaths s A = {ir e finpaths A | <S(7r)(_L) > 0}. 

Similarly, maxpaths A (s) is the set of maximal paths of A under S starting in s. 

We now define the behaviour of a PA A under a scheduler S. As schedulers 
resolve all nondeterministic choices, this behaviour is fully probabilistic. We can 
therefore compute the probability that, starting from a given state s, the path 
generated by S has some finite prefix ir. This probability is denoted by P As {n)- 

Definition 6 (Path probabilities). Let A be a PA, S a scheduler for A, and 
s a state of A. Then, we define the function P A s : finpaths A (s) — > [0, 1] by 

Pls(s) = 1; Pa,s(^ t) = PZJtt) ■ S(n)(last(TT), a,y) ■ 

Based on these probabilities we can compute the probability distribution 
F A {s) over the states where a PA A under a scheduler S terminates when starting 
in state s. Note that F A {s) is potentially substochastic (i.e., the probabilities do 
not add up to 1) when S allows infinite behaviour. 

Definition 7 (Final state probabilities). Let A be a PA and S a scheduler 
for A. Then, we define the function F A : S — > Distr*(5) by 

Fi(s) = {s'^ Yl i3»-S(7r)(-L)KeS} V.S6S. 

TT^maxpaths^ (s) 
last(7r) — s' 
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3 Branching probabilistic bisimulation 

The notion of branching bisimulation for non-probabilistic systems was first in- 
troduced in [17]. Basically, it relates states that have an identical branching 
structure in the presence of r-actions. Segala defined a generalisation of branch- 
ing bisimulation for PAs [15], which we present here using the simplified defi- 
nitions of [16]. First, we intuitively explain weak steps for PAs. Based on these 
ideas, we then formally introduce branching probabilistic bisimulation. 

3.1 Weak steps for probabilistic automata 

As r-steps cannot be observed, we want to abstract from them. Non-probabilis- 
tically, this is done via the weak step. A state s can do a weak step to s' under 
an action a, denoted by s s', if there exists a path s ^ s\ ^ . . . ^> s n s' 
with n > (often, also r-steps after the a-action are allowed, but this will 
not concern us). Traditionally, s => s' is thus satisfied by an appropriate path. 

In the probabilistic setting, s => /i is satisfied by an appropriate scheduler. 
A scheduler S is appropriate if for every maximal path ir that is scheduled from s 
with non-zero probability, trace(ir) = a and the a-transition is the last transition 
of the path. Also, the final state distribution F%(s) must be equal to fi. 

Example 8. Consider the PA shown in Figure 1(a). We demonstrate that s => fi, 



with fi — {si i-> 3j , s 2 i-> ^j, s 3 i-> ^, s 4 i-> |j, s 5 i-> ^}. Take the scheduler S: 

S{s) = {(s, r, l t2 ) ^ 2/3, (s, t, l ts ) i ^ 1/3} 
S{t 2 ) = {(t 2 ,o,l ai ) i ^ l/2,(i 2 ,r,l t4 ) ^ 1/2} 
5(t 3 ) - {(*3, a, {s 4 ^ 1/2, s 5 ^ 1/2}) ^ 1} 
S(t 4 ) = {(*4,a,l S2 ) h-> 3/4, (t 4 ,a,{s 2 1/2, s 3 i-> 1/2}) i-> 1/4} 
5(ii) = = <S(s 2 ) = S(s 3 ) = S(s 4 ) = 5(a 5 ) = 1± 



Here we used <S(s) to denote the choice made for every possible path ending in s. 

The scheduler is depicted in Figure 1(b). Where it chooses probabilistically 
between two transitions with the same label, this is represented as a combined 
transition. For instance, from t 4 the transition (t 4 ,a, {s 2 i-> 1}) is selected with 




(a) A PA A. 




Fig. 1. Weak steps. 
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probability 3/4, and (i 4 ,a, {s 2 i-> 1/2, S3 i-> 1/2}) with probability 1/4. This 
corresponds to the combined transition (£4, a, {s 2 i-> 7/8, S3 1/8}). 

Clearly, all maximal paths enabled from s have trace a and end directly after 
their a-transition. The path probabilities can also be calculated. For instance, 

pU sT,{t ^ 1} ^ {t - 1} t* a ' {s ^ 1} «o = (i • 1) • a • !) • (i • 1) = m 

PL(s TAt ^ 1} t 2 T ' {t i? 1} t 4 "' {s2 " 1/ i S3 " 1/2} s 2 ) = (I ■ 1) ■ (I • 1) ■ (I • I) = ± 

As no other maximal paths from s go to s 2 , Fj^(s)(s 2 ) = ^ + = ^ = m( s 2)- 
Similarly, it can be shown that F^(s)(si) — /Li(sj) for i G {1, 3, 4, 5}, so indeed it 
holds that F%(s) = p. □ 



3.2 Branching probabilistic bisimulation 

Before introducing branching probabilistic bisimulation, we need a restriction 
on weak steps. Given an equivalence relation R, we let s =^r fi denote that 
(s, t) G R for every state t before the a-step in the tree corresponding to s ^> fi. 

Definition 9 (Branching steps). Let A = (S, s°, L, A) be a PA, seS, and R 
an equivalence relation over S. Then, s =^r fi if either (1) a = r and [i = l s , 
or (2) there exists a scheduler S such that F%(s) = fi and for every maximal 
path s ai -^> 1 si a2 -^> 2 s 2 a3 ~^> 3 . . . a "-^>" s„ G maxpaths^(s) it holds that a n — a, as 
well as a, = r and (s, Sj) G i? for all 1 < i < n. 

Definition 10 (Branching probabilistic bisimulation). Let A= (S, s°, L, A) 

be a PA, then an equivalence relation R C S x S is a branching probabilistic 
bisimulation for A if for all (s,t) G R 

s /j, implies 3// G Distr(S) . t =^b, t 1 ' A \i =r p! . 

We say that p,q G S are branching probabilistically bisimilar, denoted p ^b P q, 
if there exists a branching probabilistic bisimulation R for A such that (p,q) G R. 

Two PAs are branching probabilistically bisimilar if their initial states are (in 
the disjoint union of the two systems; see Remark 5.3.4 of [16] for the details). 

This notion has some appealing properties. First, the definition is robust in the 
sense that it can be adapted to using s =^r h instead of s p, in its condition. 
Although this might seem to strengthen the concept, it docs not. Second, the 
relation fc^p induced by the definition is an equivalence relation. 

Proposition 11. Let A — (S,s°,L,A) be a PA. Then, an equivalence relation 
RC S X S is a branching probabilistic bisimulation for A iff for all (s,t) G R 

s =^r fi implies 3// G Distr(S') . t ==>_r p! A /i =r p . 

Proposition 12. The relation ^b P is an equivalence relation. 

Moreover, Segala showed that branching bisimulation preserves all properties 
that can be expressed in the probabilistic temporal logic WPCTL (provided that 
no infinite path of r-actions can be scheduled with non-zero probability) [15]. 
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4 Confluence for probabilistic automata 

The reductions we introduce are based on sets of confluent T-transitions. Basi- 
cally, such transitions do not influence a system's behaviour, i.e., a confluent step 
s s' implies that s t±bp s'. Confluence therefore paves the way to state space 
reductions modulo branching probabilistic bisimulation (e.g., by giving confluent 
r-transitions priority). Note that not all r-transitions connect bisimilar states; 
even though their actions are unobservable, r-steps might disable behaviour. The 
aim of our analysis is to underapproximate which T-transitions are confluent. 

For non-probabilistic systems, several notions of confluence already exist [3]. 
Basically, they all require that if an action a is enabled from a state that also en- 
ables a confluent r-transition, then (I) a will still be enabled after taking that r- 
transition (possibly requiring some additional confluent T-transitions first), and 
(2) we can always end up in the same state traversing only confluent T-steps, no 
matter whether we started by the a- or the r-transition. 

Figure 2 depicts the three notions of confluence we will generalise [3]. They 
should be interpreted as follows: for any state from which the solid transitions 
are enabled (universally quantified), there should be a matching for the dashed 
transitions (existentially quantified). A double-headed arrow denotes a path of 
zero of more transitions with the corresponding label, and an arrow with label a 
denotes a step that is optional in case a = r (i.e., its source and target state may 
then coincide). The weaker the notion, the more reduction potentially can be 
achieved (although detection is harder). Note that we first need to find a subset 
of r-transitions that we believe are confluence; then, the diagrams are checked. 

For probabilistic systems, no similar notions of confluence have been defined 
before. The situation is indeed more difficult, as transitions do not have a single 
target state anymore. To still enable reductions based on confluence, only r- 
transitions with a unique target state might be considered confluent. The next 
example shows what goes wrong without this precaution. For brevity, from now 
on we use bisimilar as an abbreviation for branching probabilistically bisimilar. 

Example 13. Consider two people each throwing a die. The PA in Figure 3(a) 
models this behaviour given that it is unknown who throws first. The first charac- 
ter of each state name indicates whether the first player has not thrown yet (X), 
or threw heads (H) or tails (T) , and the second character indicates the same for 
the second player. For lay-out purposes, some states were drawn twice. 



i a 




(a) Weak confluence. (b) Confluence. (c) Strong confluence. 

Fig. 2. Three variants of confluence. 
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HH HT TH TT HH TH HT TT 



HH HT TH TT 



(a) The original specification. 



(b) A wrong reduction. 



Fig. 3. Two people throwing dice. 



We hid the first player's throw action, and kept the other one visible. Now, it 
might appear that the order in which the a- and the r-transition occur does not 
influence the behaviour. However, the r-step does not connect bisimilar states 
(assuming HH, HT, TH, and TT to be distinct). After all, from state XX it is 
possible to reach a state (XH) from where HH is reached with probability 0.5 and 
TH with probability 0.5. From HX and TX no such state is reachable anymore. 
Giving the r-transition priority, as depicted in Figure 3(b), therefore yields a 
reduced system that is not bisimilar to the original system anymore. □ 

Another difficulty arises when defining prob- g r ^ ^ r ^ ^ 

abilistic confluence. Although for LTSs it is /\ /i\ 



how should the model shown here be completed for the r-steps to be confluent? 

Since we want confluent r-transitions to connect bisimilar states, we must 
assure that s, t , and t are bisimilar. Therefore, ll and v must assign equal 
probabilities to each class of bisimilar states. Given the assumption that the 
other confluent r-transitions already connect bisimilar states, this is the case if 
fi =r v for R — {(s, s') | s ^»<£- s' using only confluent r-steps}. The following 
definition formalises these observations. Here we use the notation s ^ s', given 
a set of r-transitions c, to denote that s A s' and (s, r, s') G c. 

We define three notions of probabilistic confluence, all requiring the target 
state of a confluent step to be able to mimic the behaviour of its source state. In 
the weak version, mimicking may be postponed and is based on joinability (Def- 
inition 14a). In the default version, mimicking must happen immediately, but 
is still based on joinability (Definition 14b) . Finally, the strong version requires 
immediate mimicking by directed steps (Definition 16). 

Definition 14 ((Weak) probabilistic confluence). Let A = (S, s° , L, A) be 

a PA and c C {(s, a,/i) G A \ a — r, fx is deterministic} a set of r-transitions. 
(a) Then, c is weakly probabilistically confluent if R — {(s,s') | s ^»«^ s'} is 
an equivalence relation, and for every path s t and all a G L, \x G Distr(S') 



clear that a path ar should reach the same state 
as ra, for PAs this is more involved as the a- 
step leads us to a distribution over states. So, 





{{3v G Distr(S') . t' ^ v A li =r v) V (a = r A \i = R l t /)). 
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(a) Weak probabilistic confluence. 



(b) Strong probabilistic confluence. 



Fig. 4. Weak versus strong confluence. 



(b) If for every path s -^*» t and every transition s \i the above implication 
can be satisfied by taking t' = t, then we say that c is probabilistically confluent. 

For the strongest variant of confluence, moreover, we require the target states 
of fj, to be connected by direct r c -transitions to the target states of v. 

Definition 15 (Equivalence up to r c -steps). Let /j,,u be two probability dis- 
tributions, and let v = {t\ i-> pi,t 2 •— > Vii ■ ■ ■}■ Then, fi is equivalent to v up 
to r c -steps 7 denoted by fj, ^> v, if there exists a partition spt(/i) = 1+J" =1 Si such 
that n = \spt(v)\ and VI <i<n: fi(Si) — v(U) A Ms G Si : s -% U. 

Definition 16 (Strong probabilistic confluence). Let A = (S, s°,L, A) be a 
PA and c C {(s, a, n) G A \ a = r, /x is deterministic} a set of r -transitions, then 
c is strongly probabilistically confluent if for all s ^ t, a G L, fj, G Distr(S) 

s ^> n ==> ((3v e Distr(5) . t ^ v A v) V (a = r A /z = l t )) . 

Proposition 17. Strong probabilistic confluence implies probabilistic confluence, 
and probabilistic confluence implies weak probabilistic confluence. 

A transition s ^> t is called (weakly, strongly) probabilistically confluent if there 
exists a (weakly, strongly) probabilistically confluent set c such that (s,r, t) G c. 

Example 18. Observe the PAs in Figure 4. Assume that all transitions of s, 
to and t are shown, and that all Si,ti, are potentially distinct. We marked all 
r-transitions as being confluent, and will verify this for some of them. 

In Figure 4(a), both the upper T c -steps are weakly probabilistically confluent, 
most interestingly s ^ to- To verify this, first note that t ^ t is (as t 
has no other outgoing transitions), from where the a-transition of s can be 
mimicked. To see that indeed fi =r v (using R from Definition 14), observe 
that R yields two equivalence classes: C\ = {s2,ti,t 2 } and C2 = {si,t 3 }. As 
required, /u(Ci) = \ = v{C\) and ^(C 2 ) = 5 = K^)- Clearly s to is not 
probabilistically confluent, as to cannot immediately mimic the a-transition of s. 

In Figure 4(b) the upper r c -transition is strongly probabilistically confluent 
(and therefore also (weakly) probabilistically confluent). For this, t must be able 
to directly mimic the a-transition from s. Indeed, it can do so by the transition 
t v. Moreover, \i ~^ v also holds, which is easily seen by taking the partition 



Si = {si},S 2 = {s 2 ,s 3 }. 



□ 
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The following theorem shows that weakly probabilistically confluent r-tran- 
sitions indeed connect bisimilar states. With Proposition 17 in mind, this also 
holds for (strong) probabilistic confluence. Additionally, we show that confluent 
sets can be joined (so there is a unique maximal confluent set of r-transitions) . 

Theorem 19. Let A = (S, s°, L, A) be a PA, s, s' 6 S two of its states, and c 
a weakly probabilistically confluent subset of its t -transitions. Then, 

s s' implies s ^b P s'. 

Proposition 20. Letc,c' be (weakly, strongly) probabilistically confluent sets of 
t -transitions. Then, cDc' is also (weakly, strongly) probabilistically confluent. 

5 State space reduction using probabilistic confluence 

As confluent r-transitions lead from a state s to a state s' such that s' is equiv- 
alent to s (with respect to branching probabilistic bisimulation) , all states that 
can reach each other via such transitions can be merged. That is, we can take 
the original PA modulo the equivalence relation and obtain a reduced and 
bisimilar system. The next definition and theorem formally state this. 

Definition 21 (A/R). Let A = (S,s°,L,A) be a PA and R an equivalence 
relation over S, then we write A/R to denote the PA A modulo R. That is, 

A/R=(S/R, [s°] R ,L,A R ), 

with A R C S/R x L x Distr(5/i?) such that [s] R ^ r fi if and only there exists 
a state s' e [s]r such that s' // and V[t] R G S/R . n([t] R ) = J2t'e[t] B /AO- 
Theorem 22. Let A be a PA and c a weakly probabilistically confluent subset 
of its t -transitions, then (A/ « Tc » ) ±^bp A. 

The downside of this method is that, in general, it is hard to compute the 
equivalence classes according to «^h>. Therefore, a slightly adapted reduction 
technique was proposed in [3] , and later used in [4] . There, for each equivalence 
class a single representative state s was chosen in such a way that all transitions 
leaving the equivalence class are directly enabled from s. This method relies on 
(strong) probabilistic confluence, and does not work for the weak variant. 

To find a valid representative, we first look at the directed (unlabeled) graph 
G = (S, ^> ). It contains all states of the original system, and denotes pre- 
cisely which states can reach each other by taking only r c -transitions. Because 
of the restrictions on r c -transitions, the subgraph of G corresponding to each 
equivalence class [s]^has exactly one terminal strongly connected component 
(TSCC), from which the representative state for that equivalence class should be 
chosen. Intuitively, this follows from the fact that r c -transitions always lead to 
a state with at least the same observable transitions as the previous state, and 
maybe more. (This is not the case for weak probabilistic confluence, therefore 
the reduction using representatives does not work for that variant of confluence.) 
The next definition formalises these observations. 
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Definition 23 (Representation maps). Let A be a PA and c a subset of 
its t -transitions. Then, a function <j) c : S — > S is a representation map for A 
under c if 

- Va, a' G S . s ^ s' => <f> c (s) = 4> c {s'); 
-VseS . a ^> <p c (s). 

The first condition ensures that equivalent states are mapped to the same rep- 
resentative, and the second makes sure that every representative is in a TSCC. 
If c is a probabilistically confluent set of r-transitions, the second condition and 
Theorem 19 immediately imply that a £±b p 4>c (s) for every state s. 

The next proposition states that for finite-state PAs and probabilistically 
confluent sets c, there always exists a representation map. As r c -transitions are 
required to always have a deterministic distribution, probabilities are not in- 
volved and the proof is identical to the proof for the non-probabilistic case [3] . 

Proposition 24. Let A be a PA and c a probabilistically confluent subset of its 
t -transitions. Moreover, let Sa be finite. Then, there exists a function <p c : S — >• S 
such that 4> c is a representation map for A under c. 

We can now define a PA modulo a representation map <j> c . The set of states 
of such a PA consists of all representatives. When originally s fj, for some 
state a, in the reduced system <j> c (s) // where //' assigns a probability to each 
representative equal to the probability of reaching any state that maps to this 
representative in the original system. The system will not have any r c -transitions. 

Definition 25 (A/<p c )- Let A = (S,s°,L,A) be a PA and c a set of r-transi- 
tions. Moreover, let 4> c be a representation map for A under c. Then, we write 
A/cfic to denote the PA A modulo (f> c . That is, 

A/4> c = (MS),Ms°),L,A^), 

where (f> c (S) — {<pc(s) \ s g S}, and A ( /, c C (j> c (S) x L x Distr(0 c (S')) such that 
s /i if and only if a ^ t c and there exists a transition t // in A such that 
4> c {t) = a and Va' e <f> c (S) . fJ,(s') = fi'({s" e S | 4> c (s") = a'}). 

From the construction of the representation map it follows that A/<j> c ^bp -4 
if c is (strongly) probabilistically confluent. 

Theorem 26. Let A be a PA and c a probabilistically confluent set of r-transi- 
tions. Also, let 4> c be a representation map for A under c. Then, (A/(f> c ) ^bp A. 

Using this result, state space generation of PAs can be optimised in exactly the 
same way as has been done for the non-probabilistic setting [4] . Basically, every 
state visited during the generation is replaced on-the-fly by its representative. In 
the absence of r-loops this is easy; just repeatedly follow confluent r-transitions 
until none are enabled anymore. When r-loops are present, a variant of Tarjan's 
algorithm for finding SCCs can be applied (see [3] for the details). 
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6 Symbolic detection of probabilistic confluence 

Before any reductions can be obtained in practice, probabilistically confluent 
r-transitions need to be detected. As our goal is to prevent the generation of 
large state spaces, this has to be done symbolically. 

We propose to do so in the framework of prCRL and LPPEs [9], where 
systems are modelled by a process algebra and every specification is linearised 
to an intermediate format: the LPPE (linear probabilistic process equation). 
Basically, an LPPE is a process X with a vector of global variables g of type G 
and a set of summands. A summand is a symbolic transition that is chosen 
nondeterministically, provided that its guard is enabled (similar to a guarded 
command). Each summand i is of the form 

^ Ci(g, di) => a,i(g, di) ^ fi(g, di,ei) : X(ni(g, di,ei)). 
di-Di t .:/•;. 

Here, di is a (possibly empty) vector of local variables of type Di, which is 
chosen nondeterministically such that the condition a holds. Then, the action 
a,i(g, di) is taken and a vector of type Ei is chosen probabilistically (each 
with probability fi(g, di, ei)). Then, the next state is set to rii(g, di, ei). 

The semantics of an LPPE is given as a PA, whose states are precisely all 
vectors g 6 G. For all g £ G, there is a transition g [i if and only if for at 
least one summand i there is a choice of local variables di G Di such that 

Ci(g,di) A cii(g,di) = a AVei e Ei . /j,(ni(g,di,ei)) = f t (g,di,e'i). 

e'iEEi 

ni(g,di,ei)=ni(g,di,e'i) 

Example 27. As an example of an LPPE, observe the following specification: 
X(pc: {1,2})= P° = 1 output(n) ^ § : X(i) (1) 

n:{l,2,3} i:{l,2} 

+ pc=2=^beep^ 1: X(j) (2) 

The system has one global variable pc (which can be either 1 or 2), and consists of 
two summands. When pc = 1, the first summand is enabled and the system non- 
deterministically chooses n to be 1, 2 or 3, and outputs the chosen number. Then, 
the next state is chosen probabilistically; with probability i it will be X(l), and 
with probability | it will be X(2). When pc = 2, the second summand is enabled, 
making the system beep and deterministically returning to A(l). 

In general, the conditions and actions may depend on both the global vari- 
ables (in this case pc) and the local variables (in this case n for the first sum- 
mand) , and the probabilities and expressions for determining the next state may 
additionally depend on the probabilistic variables (in this case i and j). □ 

Instead of designating individual r-transitions to be probabilistically conflu- 
ent, we designate summands to be so in case we are sure that all transitions they 
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might generate are probabilistically confluent. For a summand i to be confluent, 
clearly ai(g, di) = t should hold for all possible values of g and di. Also, the next 
state of each of the transitions it generates should be unique: for every possible 
valuation of g and di, there should be a single ej such that fi(g,di,ei) = 1. 

Moreover, a confluence property should hold. For efficiency, we detect a strong 
variant of strong probabilistic confluence. Basically, a confluent r-summand i has 
to commute properly with every summand j (including itself). More precisely, 
when both are enabled, executing one should not disable the other and the order 
of their execution should not influence the observable behaviour or the final state. 
Additionally, i commutes with itself if it generates only one transition. Formally: 



(0,(9, di) A Cj(g, dj)) ->■ (i = j A rii(g, di) = rij(g, dj)) V 

/ Cj(rii(g,di),dj) A Ci(rij(g,dj,ej),di) \ 
A a,j(g,dj) = aj(ni(g,di),dj) 
A fj{g, dj,ej) = fj(rii(g, di), dj,ej) 

\ A nj (rii (g,di), dj ,ej) = n; (nj (g, dj ,&,-),<**)/ 



(1) 



where g, di, dj and ej universally quantify over G, Di, Dj, and Ej, respectively. 
We used rii(g, di) to denote the unique target state of summand i given global 
state g and local state di (so ej does not need to appear). 

As these formulas are quantifier-free and in practice often either trivially false 
or true, they can easily be solved using an SMT solver for the data types involved. 
Note that n 2 formulas need to be solved (n being the number of summands) ; the 
complexity of this depends on the data types. In our experiments, all formulas 
could be checked with fairly simple heuristics (such as validating them vacuously 
by finding contradictory conditions, or by detecting that two summands never 
use or change the same variable). 

Theorem 28. Let X be an LPPE and A its PA. Then, if for a summand i we 
have Vg e G,di G Di . ai(g,di) = r A 3e; e Ei . fi(g,di,ei) = 1 and for- 
mula (1) holds, the set of transitions generated by i is probabilistically confluent. 



7 Case study 

To illustrate the power of probabilistic confluence reduction, we applied it on 
the leader election protocol introduced in [9]. This protocol, between two nodes, 
decides on a leader by having both parties throw a die and compare the results. 
In case of a tie the nodes throw again, otherwise the one that threw highest will 
be the leader. We hid all actions needed for rolling the dice and communication, 
keeping only the declarations of leader and follower. The complete model in 
LPPE format, consisting of twelve summands, can be found in Appendix B. 

In [9] we showed the effect of dead-variable reduction on this system. Now, 
we apply probabilistic confluence reduction both to the LPPE that was already 
reduced in this way (leaderReduced) and to the original one (leader). To do 
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Table 1. Applying confluence reduction to two leader election protocols. 





Original 


Reduced 


Visited 


Running time 


Specification 


States 


Trans. 


States 


Trans. 


States 


Trans. 


Before 


After 


leader 


3763 


6158 


1399 


1922 


1471 


4022 


0.49 sec 


0.35 sec 


leaderReduced 


1693 


2438 


589 


722 


661 


1382 


0.22 sec 


0.13 sec 


leader-2-6 


535 


710 


199 


212 


271 


512 


0.15 sec 


0.18 sec 


leader-2-36 


18325 


23690 


6589 


6662 


9181 


17102 


13.23 sec 


7.38 sec 


leader-3-12 


161803 


268515 


56839 


68919 


84059 


158403 


70.31 sec 


39.50 sec 


leader-3-18 


533170 


880023 


188287 


226011 


276692 


518991 


471.42 sec 


343.92 sec 


leader-3-19 


out of 


memory 


220996 


264996 


324544 


608433 




379.19 sec 


leader-4-5 


443840 


939264 


128553 


200312 


206569 


418632 


467.69 sec 


93.36 sec 



this automatically, we implemented a prototype tool in Haskell for confluence 
detection and reduction using heuristics 1 , relying on Theorem 28. 

We used confluence information when generating the state space, apply- 
ing Theorem 26. As the specification does not contain loops of confluent r- 
summands, we could from each state repeatedly execute confluent r-summands 
until reaching a state that does not enable any confluent r-summand anymore, 
adding only this state to the state space (so no detection of TSCCs was needed). 

The results, obtained on a 2.4 GHz, 2 GB Intel Core 2 Duo MacBook, are 
shown in Table 1; we list the size of the original and reduced state space, as well 
as the number of states and transitions that were visited during its generation 
using confluence. Probabilistic confluence reduction clearly has quite an effect on 
the size of the state space, as well as the number of visited states and therefore 
the running time. Notice that it nicely works hand-in-hand with dead-variable 
reduction. Applying both, we reduced by almost an order of magnitude. 

We also modeled another leader election protocol that uses asynchronous 
channels and allows for more parties (Algorithm B from [6]). We looked at ei- 
ther 2, 3 or 4 parties, who throw either a normal die or one with more or less sides 
(5, 12, 18, 19, 36). Confluence reduction reduces the state space by about 65%, 
and the number of visited states (and therefore the running time) by about 50%. 
With probabilistic POR, comparable results were obtained for similar proto- 
cols [8]. As was to be expected, detecting confluence mostly pays off for the 
larger state spaces. Still, confluence detection only took a fraction of a second 
for each system; practically all the effort is in the state space generation. From 
about 180000 states swapping occurs, explaining the excessive growth in running 
time. Confluence reduction clearly allows us to do more before reaching this limit. 

8 Conclusions 

This paper introduced three new notions of confluence for probabilistic au- 
tomata. We first established several facts about these notions, most importantly 
that they identify branching probabilistically bisimilar states. Then, we showed 
how probabilistic confluence can be used for state space reduction. As we used 

1 The implementation, case studies and a test script can be downloaded from 
http: //fmt . cs .utwente .nl/~timmer/papers/tacas2011 .html. 
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representatives in terminal strongly connected components, these reductions can 
even be applied to systems containing r-loops. We discussed how confluence can 
be detected in the context of a probabilistic process algebra with data by prov- 
ing formulas in first-order logic. This way, we enabled on-the-fly reductions when 
generating the state space corresponding to a process-algebraic specification. A 
case study illustrated the power of our methods. 
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A Proofs 

A.l Proof of Proposition 11 

Proposition 11. Let A = (S,s°,L,A) be a PA. Then, an equivalence relation 
R C S x S is a branching probabilistic bisimulation for A iff for all (s, t) G R 

s =^r p implies 3p' e Distr(S) . t =^_r p! A p =r p! . 

Proof. If every weak step s =^>r p can be mimicked, then also every step s p 
can be mimicked. After all, from s p it follows that s =^r p for any R (by 
taking a scheduler that chooses the transition (s, a, p) with probability 1 from s, 
and chooses _L with probability 1 for all other histories). Therefore, the definition 
given in this proposition is at least as restrictive as the original definition. 

Conversely, we show that when every step s p can be mimicked, then also 
every weak step s =^>r p can be mimicked. When a — r and \i = l s this weak 
step can be mimicked trivially by t =^>r It- Therefore, from now on we assume 
that there exists a scheduler S such that F^(s) = /j,, and for every maximal path 
s ^ Sl ^ S2 ^ . . . ^ Sn ^ maxpaths A (s) 

— en = t and (s, Sj) 6 R for all 1 < i < n; 

- a n = a. 

As every single transition can be mimicked by t, we can define a scheduler S' 
that mimics every choice of S. So, when S chooses the transition (s, a\, fj,i) with 
probability p, we let S' schedule the transitions necessary for t ==h'R fi[ (with 
A*i =R Mi) W1 *h probability p. That is, when for instance t t\ and t ^ t 2 
should both be assigned probability 0.5 to yield t =^_r fj,[, we let S' choose 
them with probability 0.5p. This way, with probability p the tree starting from t 
reaches a distribution over states that is i?-equivalent to \i. As we can then again 
mimic the transitions of S from there, and this can continue until the end of each 
maximal path of S, we obtain a scheduler S 1 for which (t) = p! with ji =r [i! . 
Moreover, all the states visited before the a-actions in the tree starting from t 
also remain in the same R equivalence class because of the restrictions of the 
=^r relation and the fact that the mimicked steps should yield an i?-equivalent 
distribution. Therefore, indeed t =^r pi! A p =r p! . □ 
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A. 2 Proof of Proposition 12 

Before proving Proposition 12, we first provide a definition and two lemmas. 

Definition 29 (Relation composition). Given two relations R\ and Ri over 
a set S, we use i? 2 o R x to denote their composition: i? 2 ° Ri — {(x, z) £ S x S \ 
3yeS. (x,y) £ Ri,(y,z) e R 2 }. 

Lemma 30. Let A = (S, s°,L, A) be a PA, s £ S, and R an equivalence relation 
over S. Let R' CS x S such that R' D R. Then 

s /x implies s ===>_r' /x 

Proof. Let s =>r. If a = t and /x = l s then by definition s ==>_r' /x for any R' , 
so from now on we assume the other case: there exists a scheduler S such that 
F^(s) = /x, and for every path s ° 1 -^> 1 s\ ° 2 -^> 2 s 2 a '^> 3 . . . a "-^>" s n £ maxpaths A (s) 

— a,i = t and (s, s^ £ R for all 1 < i < n; 

- a n = a. 

Now it is easy to see that the same scheduler proofs the validity of s =^>r> t 1 - Af- 
ter all, the only thing that has to be checked when changing R is that (s, Sj) £ R' 
still holds for all 1 < i < n. However, as (s, Sj) G R is assumed and R' D R', this 
is immediate. □ 

Lemma 31. Let A — (S,s a ,L,A) be a PA. Let R C S x S be an equivalence 
relation such that for all (s, t) £ R it holds that 

s =^r \x implies 3/x' £ Distr(S) . t =^>r \i! A /x =r p! . 

Then, for every equivalence relation R' C S x S such that R' D R, it holds that 

s =^>r' jU implies 3/x' G Distr(5) . t /x' A /x =_r' /x'. 

Proof. Let .4 = (S, s°,L,Zi) be a PA, and let i? C 5 x 5 be an equivalence 
relation such that for all (s, t) £ R it holds that 

s =^7? /i implies 3/x' G Distr(S) . t =^>r p! A /x =r // 

By Proposition 11 it follows that for all (s,t) £ R it holds that 

s /x implies 3/x' G Distr(S) . t =^>r /x' A it =r /x' 

Let J?' C S x 5 be an equivalence relation such that R' D R. Then, by Lemma 30 
it also holds that 

s /x implies 3/x' G Distr(5) . t ==>ri /x' A /x =r /x' 
Using Proposition 5.2.1.1 and 5.2.1.5 from [16] we obtain that 

s ^ pi, implies 3/x' G Distr(5) . t =^>r> /x' A /x /x' 
Now, applying Proposition 11 again, this lemma follows. □ 
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Proposition 12. The relation ±^bp is an equivalence relation. 

Proof. Reflexivity of t± bp trivially holds; the identity relation {(s, s) | s G S} 
can be used as the branching probabilistic bisimulation. 

For symmetry, assume that p t±bp Q- Then, there must exist a branching 
bisimulation R C S x S such that (p, q) G S. As every branching bisimulation is 
an equivalence relation, also (q,p) G S, so also q ^b P P- 

For transitivity, let p t^bp q and q ±^bp v. Then, using Proposition 11, there 
exists an equivalence relation R\ C S x S such that (p,q) 6 and for all 
(s, f) G i?i it holds that 

s =^r 1 p implies 3// G Distr(5) . t =^r 1 // A p =r 1 /i'. 

Similarly, there exists an equivalence relation R 2 C S x S such that (g, r) G i?2, 
and for all (s, t) G i?2 it holds that 

s ==^r 2 /i implies 3// G Distr(S) . i =^r 2 p' A /i =# 2 //. 

Wc define i? 3 = (R 2 o fij) U (i?i o R 2 ), and let i? be the transitive closure 
of R 3 . We first prove that R is an equivalence relation by showing (1) reflexivity, 

(2) symmetry, and (3) transitivity. 

(1) As R\ are R 2 are equivalence relations, they are reflexive; thus, for every state 
s G S it holds that (s,s) G R\ and (s,s) G R 2 . Therefore, (s,s) £ R 2 o R 1 
and thus (s, s) G R. 

2) First observe that when (x, z) G i?2 ° Ri, then there must be a y G 5 such 
that (x, y) G R\ and (y,z) G R 2 , and therefore by symmetry of R\ and i?2 
also (y, x) G i?i and (z,y) G i?2, and thus (z, x) G i?i o i?2- 
Now let (s,f) G R. Then there is an integer n > 2 such that there exists 
a sequence of states s\, s 2 , ■ ■ ■ , s n such that s\ = s and s n — t, and for all 
1 < i < n it holds that (sj,Sj+i) G (R 2 o or (sj,Sj + i) G o i? 2 )- 
By the observation above we can reverse the order of the states, obtaining 
the sequence s n , s n -i, ■ ■ ■ , Si such that still s n — t and s\ = s, and for all 
1 < i < n it holds that (sj, Sj+i) G (i?2 or (sj, Sj+i) G o R 2 ). To be 
precise, when (sj,Sj + i) G (i?2 -Ri), then (sj + i,Sj) G o R 2 ), and when 
(sj,Sj + i) G (Ri o _R 2 ), then (sj + i,s,) G (i?2 ° The sequence obtained 
in this way proves that (t, s) G R. 

(3) By definition. 

We now prove that p t±b P r by showing that (p, r) G R 1 and that for all (s, w) G R 
it holds that 

s implies 3// G Distr(S) . u =^>,r // A p =r [i 

As (p, q) G i?i and (q, r) <E R 2l it follows immediately that (p, r) e R 2 o R x and 
therefore indeed (p, r) G R. 

We prove the second part with induction to the number of transitive steps 
needed to include (s, u) in R. 
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Base case. Let (s, u) G R because (s, u) G Ri o i? x (the case where (s, u) e R 
because (s,u) G i?i o i? 2 can be proven symmetrically). This implies that 
there exists a state A such that (s,i) G J?i and (A, ix) G i?2- Let s ^ fi. Then 
we know that there exists a // G Distr(5) such that A ==>r 1 /x' and /x =_r x //. 
By Lemma 30 it follows that t ===>xj //, and using Proposition 5.2.1.1 and 
5.2.1.5 from [16] we see that /i =r p! . 

As R D i?2, we know by Lemma 31 that for all (t, u) G R2 it holds that 

A =^>r A*' implies 3/i" G Distr(5) . u =^r A*" ^ A*' =x? A 4 "- 

We thus showed that s /x implies t =^>r \i' (with /x =r /i'), and that 
A ===>_r /i' implies u ===>_r /i" (with /x' =r /i"). Therefore, it follows that if 
s /i, indeed there exists a a*" G Distr(S) such that u =^r fi". As =r is 
an equivalence relation, /x /x" follows by transitivity. 
Induction hypothesis. Let (s, t) G R by fc transitive steps. Then, 

s A 1 implies 3/x' G Distr(5) . A ==5>r ax' A /j =r /x'. 

Inductive step. Let (s, it) G R by fc + 1 transitive steps. That is, there exists 
some A such that (s, t) G i? by means of Ac transitive steps, and either (A, xx) G 
R2 o i?i or (A, u) E Ri o R 2 . We then need to show that 

s /x implies 3/x" G Distr(5) . u =^>r A 4 " A A* =fl A 4 "- 

By the induction hypothesis we already know that s /x implies A =^=>,r a*' 
for some fi' =r A*- Moreover, using Proposition 11 and the same reasoning as 
for the base case, we know that t =^r [i! implies that u =^>r /x" for some 
/x" =r ax. Therefore, by transitivity of =_r the statement holds. □ 



A. 3 Proof of Proposition 17 

Lemma 32. Let A be a PA, c C {(s, a, /x) G Z\ | a = r, /x is deterministic} a set 
of weakly probabilistically confluent r -transitions, andR — {(s,s') \ s ^»«^ s'}. 
Then, fj, v implies /x =r /x. 

Proof. Let .A be a PA, c C {(s,a, ax) G Z\ | a = t, zx is deterministic} a set of 
r-transitions. Moreover, assume that /1 ^> v. 

Thus, denoting v by v = {Ai i-> pi,t2 i-> P2, • • • }, there exists a partition 
spt(Ax) = l+J™ =1 «5, such that rx = |spt(z/)| and VI < i < n: n{Si) = u(U) A 
VseS,:s^ U. 

Now let R 1 be the smallest equivalence relation that relates the states of 
every set Si to each other and to their corresponding Aj. That is, for every Si 
and for all s, s' G Si it holds that (s, s') G R' and (s, Aj) G R'. 
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By definition R' is an equivalence relation, and clearly /j, = R i v. After all, for 
every ij it holds that 



v([U]r') = "{{tj I {ti,tj) e R'}) = Yl v ^)= E ^ S i) 



l<j<n l<j<n 



l<j<n 

VseSj . (u,s)eR' 

Now let R = {(s, s') | s s'}. A simple tiling argument shows that 

R is transitive, and reflexivity and symmetry are trivial. Therefore, R is an 
equivalence relation. Moreover, as s -^A U implies s -^»«^- ti, and for every 
s,s' £ Si we have s -^»«^- s' since they can join at ti, clearly R also relates 
the states of every set Si to each other and to their corresponding ti. Since R' 
is the smallest equivalence relation having this property, it follows that RD R' . 
Because of this, fi = R > v implies [i = R v (using Proposition 5.2.1.1 and 5.2.1.5 
from [16]), which is what we wanted to show. □ 



Proposition 17. Strong probabilistic confluence implies probabilistic conflu- 
ence, and probabilistic confluence implies weak probabilistic confluence. 

Proof. Let A be a PA and c C {(s,a, /x) £ A \ a = t, \i is deterministic} a 
strongly probabilistically confluent set of r-transitions. Then, for every transition 
s ^> t and all a £ L, /x £ Distr(5) it holds that 

s ±> fi ==>• ((3u £ Distr(S) J^i/A/i^i/) V(a = rA/i = l t )) . 

Now let R — {(s, s') | s -^»«^- s'}. As stated in the proof of Lemma 32, R is an 
equivalence relation. 

We need to proof that for every path s ^» t it holds that 

s ^ fi => ({3v £ Distr(S) . t ^ v A \i = R v) V (a = r A /x = R l t )) . 

So, let s -^A ti . . . t n be such a path, and assume strong probabilistic 
confluence. Let s ^ xt, and first assume that a ^ t. Then, by definition of 
strong probabilistic confluence it must hold that t\ /txi such that /tx ~^ /xi, 
and therefore £2 M2 such that /xi ~^ /i2, and so on, until i„ n„ such that 
Mn-i ~^ Mn- By Lemma 32 and transitivity of =_r, it follows that /j /tx„. 
So, the first disjunct of the formula we needed to prove holds (note that for the 
empty path s it also holds by reflexivity of = R ). 

Now assume that a = t. If /x ^ l tl , then the situation is the same as above. 
If /i = Itj, then it follows that /x l tn since ti -^*> t„ and thus (t\,t n ) £ R. 
So, the second disjunct of the formula we needed to prove holds. 

So, in both cases c is probabilistically confluent. 

The fact the probabilistic confluence implies weak probabilistic confluence is 
immediate from the definition, as the former is a restriction of the latter. □ 
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A. 4 Proof of Theorem 19 



Before proving Theorem 19, we first provide an important lemma. 

Lemma 33. Let A = (S,s a ,L,A) be a PA and s,t £ S. Moreover, let c be a 
weakly probabilistically confluent set of t -transitions of A. Then, 

s ^»«^- t if and only if s t. 

Proof. Let s ^»«^ t. Then, by definition there must exist a state s'eS such 
that s ^» s' and t ^» s' . Therefore, it immediately follows that s t. 

Now let s «^-» t. Then, there must be path such as s ^> sq ^> s\ -f^ 
s 2 <^ s 3 ^ S4 <^ t. Potentially (as is the case here) the path contains a 
fragment of the form Sj -f^ 2 - Sj+i Sj + 2- Clearly this violates the conditions for 
the path to show that s -^»«^- t, as the arrows point in the wrong direction. 
Also, note that a path without such a fragment does prove that s ^»«^- t. 
Therefore, we will show that in any path that can be used to show s t we 
can eliminate these kind of fragments, obtaining a path that proves s -^»«^- t. 
When Si s i+1 ^ s i+2 , then by definition of weak probabilistic confluence 
cither (1) Sj = s i+2 , or (2) there exists a state t such that s i+2 ^» t and 
Si c » « t. 

In case (1), the whole fragment can just be reduced to the state Sj, indeed 
eliminating the bad fragment. In case (2), assume that t is satisfied 

by the path Si ^ t n ^ . . . ^ t n ^ t' <^ t^ <^ . . . <^ t' n <^ t, and that 
Si+2 i is satisfied by the path s i+2 t^ +1 . . . ^ t' m ^ t. Then, the 
whole fragment can be reduced to Sj to • • • ^> t n ^> t' i' x . . . 
tjj^tf^ t' m <r^- . . . <r^- t' n+1 s i+2} which is of the correct form. 

Repeating this for all bad fragments, a path proving s Tc » u Tc t appears. □ 

Theorem 19. Let A = (S, s°,L, A) be a PA, s,s' £ S two of its states, and c a 
weakly probabilistically confluent subset of its r-transitions. Then, 

s «^h> s' implies s ^b P s' . 

Proof. Let A — (S, s°, L, A) be a PA and c a weakly probabilistically confluent 
set of r-transitions. We prove that s ^»«^ s' implies s t±b P s'. Clearly, when 
this hold also s ^ s' implies that s t±bp s'. Then, as ±^bp is an equivalence 
relation, the theorem follows. 

Let s, s' £ S such that s s'. To prove that indeed s *=*bp s', we show 

that R = {(s, t) | s ^»<&- t} is a branching probabilistic bisimulation. Obviously 
(s, s') £ R. Lemma 33 and the fact that is an equivalence relation imply 
that R is an equivalence relation. 

To show that R is a branching probabilistic bisimulation, let (s, t) £ R be an 
arbitrary pair of states in R. We prove that 

s [i implies 3fi' £ Distr(5) . t ==$*r u! A =r p! . 

Let u be the joining state of s and t, i.e., s u and t ^» u. Let s fi. We 
make a case distinction based on whether a / t or « = t. 
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— Assume that a ^ r. From the definition of weak probabilistic bisimulation 
it immediately follows that 

3v! e S . u u A 3v G Distr(S) . u' ^ v A fj, = R v 

Now let S be a scheduler choosing with probability 1 the transitions from t 
corresponding to the path t ^» u'. Then, let S choose u' v with proba- 
bility 1, followed by _L with probability 1. Clearly the final state distribution 
of S is v, and indeed \i = R v by definition of weak probabilistic conflu- 
ence. Moreover, as the scheduler only follows T c -transitions before it selects 
u' v, the branching condition is satisfied. 

— Assume that a = r. From the definition of weak probabilistic bisimulation 
it then follows that 

3v! G S . u ^ v! A ((3u G Distr(S) . u ^ v A fj, = R u) V (n= R l u >)) 

When the first disjunct is satisfied the above reasoning applies, so from now 
on assume that 3u' e S . u ^» v! A yu = R 1 u >. If t = u', then t =>b. V 
is satisfied by the first clause of the definition of branching probabilistic 
bisimulation as a = t and /i = R l u > = l t . If t ^ u', then the transition can be 
mimicked by the scheduler choosing with probability 1 the transitions from t 
corresponding to the path t ^» v! and then choosing _L with probability 1. 
Clearly the final state distribution of S is l u >, and indeed /i = R t u > by the 
assumption we made. Moreover, as the scheduler only follows r c -transitions, 
the branching condition is satisfied. □ 

A. 5 Proof of Proposition 20 

Proposition 20. Let c, c' be (weakly, strongly) probabilistically confluent sets 
of t -transitions. Then, c U c' is also (weakly, strongly) probabilistically confluent. 

Proof. Let A be a PA and c C {(s, a, fi) G A \ a — r, /x is deterministic} a weakly 
probabilistically confluent set of r-transitions. Then, for every path s ^» t and 
all a G L, fi G Distr(S') it holds that 

s ^ n => 3t' e S . t ^» t' A 

((3u G Distr(S) . t' ^ v A \i = R v) V (a = r Au,= R l t ')) 

where R — {(s, s') \ s ^»<&- s'} (and R is an equivalence relation). 

Let c' be a different weakly probabilistically confluent set of T-transitions. 
Then, for every path s t and all a G L, \i G Distr(5) it holds that 

s ^ ft =^ 3t' G S . t ^ t' A 

((3u G Distr(S) . t' ^ v A [i = RI v) V (a = r A (j, = R > l t >)) 

where R' — {(s, s') \ s ^L»<±^L s'} (and R is an equivalence relation). 
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Now, taking c" — c U c', for every path s t and all a e L, fi G Distr(5) 
it should hold that 



s -2> n =^3t' eS .t^t' A 

((3u G Distr(S) . t' A v A \i = R » v) V (a = r A (i = w , l t ,)) 

where R" = {(s,s') \ s s'}. The fact that R" is again an equivalence 

relation follows easily from the restrictions on the r c - and r c /-steps. Moreover, 
the required implication follows from a common tiling argument. 

A similar argument can be given for probabilistically confluent sets, and for 
strongly probabilistically confluent sets. □ 

A.6 Proof of Theorem 22 

Theorem 22. Let A be a PA and c a weakly probabilistically confluent subset 
of its T-transitions, then (_4/«^>) ^b P A. 

Proof. Theorem 19 already showed that all states that can reach each other via 
T c -transitions are branching probabilistically bisimilar. It is well known that such 
states can therefore be merged, preserving branching probabilistic bisimulation. 

The equivalence relation R needed to show this relates the states of the 
disjoint union of A and {A/ **^>) in such a way that every equivalence class 
contains precisely one of the states \s] t c of (A/«^) and all the states s' in A 
such that s' G \s] t„ . Clearly, (s°, [s ]^^) G R- In the remainder of the proof 
we omit the subscript of equivalence classes \s] t c , as they are always the same. 

Now, let {si, S2, ■ ■ ■ , s n , [si]} be one of the equivalence classes of R. Because of 
Theorem 19 all the states s\, 82, ■ ■ ■ , s n are branching probabilistically bisimilar, 
so we only still need to show that [s{\ can mimic the behaviour of s\, S2, ■ ■ ■ ,s n 
and vice versa. 

So, assume that for instance S2 /i. Then, by definition [s\] v such that 
KM) = St'e[t] ^C*') f° r every [t] G This implies that fi =n v by the 

construction of R. 

Conversely, let [s\] fi. Then, by definition there must exist a state Si G [s] 
such that Si v and V[i] G . n([t]) = X)t'e[t] K*')- So, /U =r v. Therefore, 

Si can mimic the behaviour of [s\]. As all the states si, S2, ■ ■ ■ , s n are branching 
probabilistically bisimilar, they can all mimic each other, so therefore all states 
can mimic the behaviour of [si]. □ 

A. 7 Proof of Theorem 26 

Theorem 26. Let A be a PA and c a probabilistically confluent set of T-transi- 
tions. Also, let <j) c be a representation map for A under c. Then, (A/<p c ) ^bp A. 

Proof. Theorem 22 already showed that {A/*^*) ^bp A if c is weakly proba- 
bilistically confluent, and by Proposition 17 this also holds for probabilistically 
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confluent sets c. As ^b P is an equivalence relation by Proposition 12, it there- 
fore suffices to prove that (A/*^*) ^bp {A/<p c )- In this proof we will omit the 
subscript of equivalence classes [s] t c , as they are always the same. 

By definition every state of A/ is an equivalence class [s], and every state 
of A/(f> c is a representative <j) c (s). We define the relation R to be the reflexive 
and symmetric closure of 

{([a], Ms)) \s&S}, 

and show that it is a branching probabilistic bisimulation. Clearly R is an equiva- 
lence relation, and by definition ([s°], 4> c (s )) G R. To show that R is a branching 
probabilistic bisimulation we prove that [s] ^> /x implies that there exists a fi' 
such that 4> c {s) ==> fJb' and /i =r xx' ', and that (f> c {s) /x implies that there 
exists a // such that [s] =^> /x' and /x /x'. 

— Let [s] /i. We prove that the exists a p! such that <j> c (s) =>■ /x' and 
/x =r pi! by showing that, assuming ix([i]) — p for an arbitrary state t, there 
exists a /u' such that 4> c (s) =^> /i' and n'(cf) c (t)) — p. 

By Definition 21, there must exist a state s' G [s] in .A and a fj," such that 
s' /x" and V[i] G S/A . /x([i]) = J2t'e[t] /""(*')• That is ; m" also assigns 
probability p to the event of going to a state in the equivalence class [t] . 
Now, by definition of representatives s' ^» <fi c (s), and therefore, by definition 
of probabilistic confluence it must be the case that 

(3u G Distr(S) . <j> c (s) ^^A/i" = R , v) V (a = t /\ fj," = R , l 0c(s) ) , 

where R' = {(s, s') | s ^»<&- s'}. First assume that a ^ r. Then, in .A we 
have 4>c(s) ^ v with f =ri /x". Given the definition of R' and Lemma 33, 
this implies that v{[t\) = /x"([i]) = P- As [t] is exactly the set of all states that 
have (f> c {t) as their representative, by Definition 25 we also have (j) c {s) v' 
with v'{(j) c {t)) = p in A/<f) c - As the existence of a transition implies the 
existence of a weak step, this finishes this part of the proof. 
When a = t, either the above holds, or fi" =ri l^ c ( s ) • In the latter case, as we 
also already knew that /x" assigns probability p to the event of going to a state 
in the equivalence class [t], apparently <f> c (s) G [t] and p = 1. From c (s) G [i] 
it follows by definition that <j> c {s) — 4> c {t)- By definition of branching steps 
4>c{s) =^ l0 c (s)i and given the above indeed l^ c ( s )((/> c (£)) = p. 

— Let 4> c (s) /x, and let /x(0 c (i)) = p for some state t. We prove that there 
exists a // such that [s] => /x' and /x'([£]) = p. As </> c (s) it, there must 
exist a transition t' ix' in the original PA such that <j> c {t') = <l>c{s) and 

W G C (S) . /i(a') = p'({s" G 5 | 4>c{s") - a'}). 

So, because we assumed /x(</> c (i)) = p, it should hold that /x'({s" G S 
<t>c{s") = </>c(i)} = P- Stated otherwise, and recognising that the set of states 
with the same representative as t is precisely the set [t], we get /x'([t]) = p. 
As t' /x' such that /x'([f]) = p, and because s and t' have the same 
representative, also [s] /x" such that n"([t]) — p. Observing that a normal 
step implies a weak step, we're done. □ 
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A.8 Proof of Theorem 28 



Theorem 28. Let X be an LPPE and A its PA. Then, if for a summand i we 
have Vg £ G,di £ Di . di(g,di) = r A 3e^ £ Ei . fi(g,di,ei) = 1 and for- 
mula (1 ) holds, the set of transitions generated by i is probabilistically confluent. 

Proof. Let X be an LPPE and A its underlying PA. So, by the operational 
semantics the state set S of this PA contains precisely all vectors g £ G. 

Let i be a summand such that Vg £ G,di £ Di . at(g,di) = t A 3e^ £ 
Ei . fi(g,di,ei) = 1 and for every summand j it holds that 

(ci(g, di) A cj(g, dj)) ->■ (i = j A rii(g, di) = nj(g, dj)) V 

( Cj(rii(g,di),dj) A Ci(nj(g,dj,ej),di) \ 
A aj(g, dj) = aj(rii(g, di), dj) ( 2 ) 
A fj (g, dj ,ej) = fj {m (g,di), dj , e d ) 
\Anj(ni(g,di),dj,ej) = m(nj(g,dj,ej),di) J 

By the operational semantics, the transitions generated by i are those transitions 
g /U such that there is a choice of local variables di £ Di such that 

d(g,di) A ai(g,di) = a AVei £ Ei . /j,(ni(g,di,ei)) = fi(g,di,e'i). 

e'tEEi 
ni(g.di.ei)—rii(g,di,e f i ) 

Let c be the set containing these transitions. We prove that c is probabilistically 
confluent by showing that it is strongly probabilistically confluent, relying on 
Proposition 17. Note that the sets Cj from all confluent summands i can be 
combined into a single confluent set by Proposition 20. 

Let g yu be an arbitrary transition in c, and let d\ £ Di be the local 
variables that had to be chosen for i to generate it. So, 

Ci(g,d'i) Aaiig^'i) = a AVei £ Ei . u.(ni(g,d'i,ei)) = f l {g,d'i,e' i ). 

n.i(g,<,ei)=n i (g,d^e^) 

Because Vg £ G,di £ Di . a,(g, di) = r A 3ej £ Ei . fi(g, di, ei) = 1, it follows 
that a — t. Moreover, fi is deterministic (as it assigns probability 1 to the next 
state determined by rii(g, d^, ei), where ei is the unique element of Ei such that 
f i (g,d' i ,ei) = 1). We use g' to denote this unique target state. Thus, using the 
notation rii(g, d'A for the unique target state given the global state g and local 
variables d^, we have g 1 = rii(g,d'j). 

So, we indeed can write g /i as g ^ g' . To show that c is strongly 
probabilistically confluent it remains to show that for every transition g \x 

(3u £ Distr(S) . g' ^ v A fj, % v) V (a = r A /i = l g >) . (3) 

Let g (i be such a transition for which this needs to be shown. Let j be 
the summand from which it originates, and let dl- be the local variables that had 
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to be chosen for j to generate it. So, 

Cj(g, d'j) A a {g, d'j) = a A Ve^- G ^ . fj,(nj(g, d'^e,)) = ^ fj(g, d'j, e'j) 

nj {g , d'j , e s ) =n s (g , d' 3 , e' 3 ) 

Now, as we showed above that Cj(<7, d^) and Cj(g,d'j) hold, we can apply 
Equation (2). Therefore, we know that either (« = j A nj(g,d^) = rij(g,dj)), 
or for every e i£j it holds that 

/ Cj(g', d'j) A Ci(nj(g, d^ej),^) \ 

A Oj(fl,dj-) = ajig^d'j) 

A /j (ff, d^. , ej ) = £ (&■' , d^. , e 3 - ) 
\A njig^d^ej) = min/fad^ej),^) J 

(where we already substituted g' for rii(g, d^). 

In the first case, both g ^> </' and g /i result from the same summand. 
Therefore, a = t is immediate, and indeed \i = l g < because g' = rii(g, d'j), 
rij(g, d'j) = nj(g,d'j), and rij(g,d'j) denotes the unique target state of j given 
the local variables d'j. Therefore, the second disjunct of Equation 3 holds. 

In the second case, we know several things. First of all, Cj(g', d'j), so j is still 
enabled using the local variables d'j in state g' . Moreover, aj(g, d'j) — a,j(g' , d'j), 
so since above we already showed that dj(g,d'j) — a, it follows that taking j 
from g' using the local variables d'j also results in the action a. So, this shows 
that there indeed exists a distribution v £ Distr(S') such that g' v. 

The final part of the proofs explains that /i ~^ v. For this, we need to show 
that there exists a partition spt(/z) = l+l^ =1 S k such that n — spt(^) and VI < 
k < n: n{S k ) = v{t k ) A Vs G S k : s ^> t k . 

To sec this, assume that v = {g'^ \-t pi,g' 2 i-> p 2 , • • • }• Now, let the partition 
be given by S k = {rij{g,d'j,e'j) \ e'j G n;(nj (g, d^., e^.), d^) = g' k }. Clearly, 
by construction the partition contains precisely as many elements as spt(^). It 
is also indeed a valid partition: (1) there is no state in spt(/z) that is not present 
in at least one of the S k s, since every such state can be written as n,j(g, d'j, e'j) 
and because of the requirement that Ci(rij(g, d'j, ej), d^) each of them goes to 
at least one of the g'^s (which is also guaranteed to be in the support of v if 
it is in the support of [i, as follows from the computation below); (2) there is 
no state in spt(/i) that occurs in more than one S k , since any such state can be 
written as rij(g, d'j, e'j) and therefore precisely occurs only in the S k such that 
7ii(™i(0> d i> e jMi) = g' k . 

Furthermore, for any S k : 

H{S k ) = v({nj(g, d'j, e'j) \ e'j G E^rnirijig, d'j, e'j), d-) = g' k }) 
£ f 3 {a,d'j,e'j) 

niinjig^d'^e'^^d'^g't, 

E W,d'j,e'j) 

n i (n j (g,d' j ,e' j ),d' i )=g' h 
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E W,d'j,e'j) 

nj (g' ,d'., ej )=g' k 

= v{g' k ) 

The first equality just unfolds the construction of Sk, the second applies the 
operational semantics of summands, the third applied the requirement that 
fj(g,d'j,ej) = fj(g',dj,ej) for every ej £ Ej, the fourth applies the require- 
ment that rij(g' ,d'j,ej) = ni(rij (g , dl- , ej) , d^) for every ej £ Ej, and the last 
equality again uses the operational semantics. 

The fact that every state s £ Sk has a T c -transition to g' k follows directly from 
the requirement that Ci(rij (g, d'-,ej),d'^) for every ej £ Ej and the construction 
that m(nj(g, d'^e'-), dj) = g' k . □ 

B Case study: a leader election protocol 

The LPPE for the leader election protocol discussed in Section 7 is shown in 
Figure 5. For readability, in every summand we only show the parameters that 
are used or updated. As summations can use an existing parameter name for a 
local variable, statements such as d-2 := d_2 occur. This means that, in the next 
state, the global variable dJ2 will have the value of the local variable dJ2. We 
use the notation reset (x) to denote that a variable x is reset to its initial value. 

We will now give an idea of how confluence detection works for this LPPE. In 
fact, for the case studies we implemented a tool that does this kind of reasoning 
automatically. 

As the summands 9, 10, 11, and 12 do not have a r-action, and the target 
states of the summands 1, 2, 7, and 8 are not determined by a deterministic 
distribution, the only candidates for confluent summands are 3, 4, 5, and 6. 
It turns out that all of these are indeed confluent. To show for instance the 
confluence of summand 3, we need to prove that, for every summand j, and all 
g, di, dj, and ej: 

(ci(g, di) A Cj(g, dj)) (i = j A n;(g, di) = rij(g, dj)) V 

/ Cj(ni(g,di),dj) Aci(rij(g,dj,ej),di) \ 
A a,j(g, dj) = a/(ni(g, di), dj) 
A fj {g, dj ,ej) = fj (rii (g, di), dj , ej ) 

\A nj(m(g,di),dj,ej) = m(nj(g,dj,ej),di) J 

Note that summands 1, 5, 7, 9, and 11 can never be enabled at the same time 
as summand 3, as summand 3 requires pc-2 = 2 and these summands all require 
pc-2 to have another value. Also, 6 can never be enabled at the same time as 3 
because of their contradictory requirements on set-3. Therefore, the formula we 
need to prove holds vacuously for all these summands (as the left-hand side of 
the implication can never be true). 

So, we only still need to prove the formula for the summands 2, 4, 8, 10, and 
12, and for 3 itself. Simple observation shows that summand 3 only deals with 
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the variables pc_2, set_3, e_2, vaL3, and d_2, and that none of the summands 2, 
4, 8, 10, and 12 either uses or updates any of these variables. This immediately 
implies that summand 3 cannot disable any of these summands or the other way 
around, that summand 3 cannot influence the actions or probabilities of these 
summands, and that the state that is reached after first executing summand 3 
and then one of these summands must be identical to the state that is reached 
when doing this the other way around (given the same probabilistic choice). 
Therefore, the formula holds for all possible valuations of its parameters (by 
means of the second disjunct). 

To see that summand 3 is confluent with itself, notice that it does not have 
any local variables to choose from. Therefore, the first disjunct of the formula 
holds trivially. 

The same kind of reasoning can be applied to show that summands 4, 5, and 
6 are confluent. 



Z(val.l : { 1 
val-S : {1. 


.6},set.l : Bool,pc-2: {1.-4}, d-2 : {1..6},eS: {1-6}, 
6}, set-S : Bool, pc-4 : {1..4}, d-4 : {1..6}, e-4 : {1..6}) = 




pc-2 


= 1 => t ^ i : Z(pc-2 : = 2, d-2 := d-2, reset(e_2)) 

d_2:{1.6} 


(1) 


+ pc-4 


= 1 => r y i : Z(pc-4 := 2, d-4 := d-4, reset(e_4)) 


(2) 


+ pc.2 


= 2 A -.sei_3 t ^ 1-0: Z(pc-2 := 3, reset(e_2), val-3 := d-2, set-S := true) 
Ml} 


(3) 


+ pc-4 


= 2 A -^set-1 => t Z(vaLl := d-4, set- 1 := true, pc-4 := 3, reset(e_4)) 
Ml} 


(4) 


+ pc.2 


= 3 A set-1 => t ^ 1.0: Z(set-1 := false, pc.2 : = 4, e_2 := val-1) 
Ml} 


(5) 


+ pc-4 


= 3 A set-3 t ^ 1.0 : Z(setS := false, pc-4 := 4, e-4 := t>aL5) 
Ml} 


(6) 


+ pc.2 


= 4 A d_2 = e-2 =S> t ^ i : Z(pc_2 := 2, d-2 := d_2, reset(e_2)) 

<i_2:{i..i;} 


(7) 


+ pc-4 


= 4 A d-4 = e-4 r ^ i : Z(pc_4 := 2, d_4 := reset(e_4)) 
<i_4:{J..<;} 


(8) 


+ pc.2 


= 4 A d-2 > e-2 =S> leader(one) ^ 1.0: Z(pc_2 := 1, reset(cL2), reset(e_2)) 
Ml} 


(9) 


+ pc-4 


= 4 A gL4 > e_4 => leader(two) 1.0: Z(pc_4 := 1, reset(eL4), reset(e_4)) 
Ml} 


(10) 


+ pc-2 


= 4 A cL2 < e-2 =t- follower(one) ^ 1.0: Z(pc_2 : = 1, reset (d_2), reset (e-2)) 

Ml} 


(11) 


+ pc-4 


= 4 A d-4 < e_4 => follower(two) ^ 1.0 : Z(pc-4 := 1, reset(d_4) , reset(e_4)) 

Ml} 


(12) 



Fig. 5. The LPPE of a leader election protocol. 
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